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ABSTRACT 

The development of leadership evaluation procedures 
has been impeded by the same obstacles that have sieved the 
development of soft skills systems engineering techniques. Problems 
include the difficulty of finding a "true expert", the general lack 
of clarity concerning ultimate criterion measures, and the fact that 
leadership skills are essentially disjunctive. The assessment center 
concept is discussed briefly as a possible prototype of future 
evaluation approaches. This paper outlines soft skills systems 
engineering, who the "true expert" is, the nature of the real world, 
measuring the impact of leadership training and development, ultimate 
criteria of measurement and a disjunctive approach for a disjunctive 
concept. (A sixteen item bibliography is included.) (KF) 
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Prefatory Note 



This paper on leadership skills was presented at the 
CONARC Soft Skills Training Conference, sponsored by the 
U.S. Continental Army Command at the Air Defense 
School, Fort Bliss, Texas, on 12-13 December 1972. The 
purpose of the conference was to foster an extensive inter- 
change among CONARC representatives and invited partic- 
ipants regarding approaches to the systems engineering of 
soft skills training. 

Since joining the Human Resources Research Organi- 
zation, Dr. Jacobs has specialized in leadership research and 
leadership training development. As Director of HumRRO 
Division No. 4 at Fort Banning, Georgia, he is presently 
responsible for direction of its research program in motiva- 
tion, leadership, and both group and individual effectiveness. 



The Evaluation of Leadership Skills 



T.O. Jacobs 



The primary purpose of this paper is to discuss some of the problems involved in 
the evaluation of leadership skills. However, it will not be possible to discuss evaluation 
of leadership skills without first considering the current state of the art in their 
identification and development. 



SOFT SKILLS SYSTEMS ENGINEERING? 

This frankly provocative title was selected to draw attention to the probable source 
of currer problems in quality control of soft skills training. A basic problem, at least 
insofar as leadership is concerned, is that the systems engineering of leadership training is 
only beginning to emerge from the dark ages. As this paper will show, there is a 
substantial lack of agreement on at least the following points: 

(1) What leadership is. 

(2) What affects leadership in organizations. 

(3) The training experience that will produce improved leadership. 

(4) Satisfactory measures of leadership ability. 

Difficulties have been encountered in the systems engineering of training. 
Crawford (1) has described seven steps in the development of training, as shown in 
Figure 1. 

A case can be made that the major problem in soft skills systems engineering lies in 
the second step, analysis of the particular job. In his discussion of this step, Crawford 
says, **We are interested in the inputs to the job— the kind of stimuli and requirements 
placed on the individual--and the outputs from the job— what the man does to bring 
about system output.'* This, of course, is a reasonable assertion. The problem arises when 
one attenipts to implement this step. While Crawford's paper is not intended to be a 
"how-to-do-it," it is instructive to note that a rough count shows that only 91 words 
were allocated to describe this step. 

It appears to the present writer that systems engineering of training implicitly 
assumes that a **true expert" is to be found somewhere, and that, once located, he will 
be able to specify his inputs and outputs. There may also be an assumption that there is 
a single correct way to do the soft skills job. In the remainder of this paper, all of these 
possible assumptions will be questioned. 

WHO IS THE TRUE EXPERT? 

Without desiring to be critical of Bn interesting and innovative beginning, it is 
instructive to search momentarily fur a true expert. Whitmore (2), in discussing his 
concept of the job r^.odel, notes that 'The identification and definition of job functions 
from system characteristics and the formulation of the behavioral processes underlying 
each function requires behavioral science expertise." This suggests the possibility that the 
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Steps in the Development of Training 



Analysis of the System Fi 



Analysis 



Specification of Knowledges and Skills 



][ 

Determination of Training Objectives 



Construction of the 
Training Program 

• Programming of instruction 

• Practice materials 

• Achievement testing 



Evaluation of the Training Program 



Figure 1 

behavioral scientist may be the "true expert," at least in this complex soft skills area. 
The validity of this assumption may be illustrated by a quick comparison of T-group 
protagonists and antagonists. 

T-group (sensitivity) training is a fairly widely used form of training to which 
executives and leaders frequently are exposed in the belief that will "develop" them. 
Bradford (3), as one protagonist, describes how sensitivity training works. He concludes 
that, while" not all individuals profit equally from such training, most benefit to some 
extent and "... the majority of participants have learned and grown." 

Adair (4), on the other hand, criticizes sensitivity training on four accounts: 

(1) Such groups have no formal tasks, which means that they do not have the 
task and team maintenance responsibilities of real groups in formal organizations. Thus, 
the experience is dissimilar to the on-the-job conditions of performance, which should 
decrease transfer of training. 

(2) Several assumptions underlying this approach probably are inaccurate. An 
example is the assumption that leadership functions should be distributed equally among 
the members of the group. Adair suggests the possibility that this may well not be true 
when there are real tasks which must be achieved under difficult situations. 

(3) This approach as a universal method of leadership training is culture-bound. 
(It should be noted that this criticism is not valid if transer of training is planned only to 
the culture to which it is "bound.") 

(4) There is a tendency for group psychotherapy to invade the training scene. 
With this invasion may come "preconceptions" which may not be appropriate for leaders 
of "normal" people. 
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The above attempt to pit two schools of thought against one another is, admittedly, 
somewhat unfair. Clearly, there would be points of agreement. Howevci, the major point 
is that there clearly are differences in objectives, implied oi explicit. That is, protagonists 
would identify one set of objectives for leader training and antagonists another set. The 
differences in objectives produce differences in the training approach taken. Further, the 
criteria that would be used t evaluate the effectiveness of the training would 
be different. 

Clearly, the **true expert" may or may not be the behavioral scientist. Further, to 
the extent that he has theoretical biases, his degree of "expertness" may be com- 
promised. And I believe that no one would assert that behavioral sciences are lacking in 
theoretical biases. 

There is yet another reason why the behavioral scientist may not be the "true 
expert." Most behavioral scientists, as McGregor (5) points out, have never been required 
to perform in the role of a line executive. McGregor, without such experience, had 
believed that a leader could operate essentially as an organizational advisor, and that good 
human relations would be quite effective in relieving discord and disagreement. Experi- 
ence demonstrated to him that a leader cannot avoid the exercise of authority. The 
leader cannot delegate the responsibility which he alone must assume for the workings of 
his organization. 

This point has been made with trepidation, and with the anticipation of challenge. It 
is in fact correct that experience in the role is not necessary in order to be able to 
describe the role through rigorous observational techniques. So some behavioral scientists 
may be able to rise beyond inexperience to become "true experts." However, it is very 
difficult to observe intangibles. Leadership and executive skills are intangible. The 
necessary recourse, there* >re, is to study experimentally the impact of variation in role 
behaviors on organ izationai outcomes. 

It should be recognized that this is an extremely difficult proposition. Several 
questions immediately are raised, which bear not only on systems engineering but also 
on evaluation: 

(1) What should be used as the criterion? Should it be evaluations by 
superiors? Should it be evaluations by subordinates? Should it be some rating or objective 
measure of organizational performsmce? 

(2) What organization can be made available for experimental study? Real 
organizations exist in a world that usually is competitive. Only rarely are they available 
for experimental study. (And they also are hard to simulate.) 

(3) How much time can be devoted to experimental study? The time required 
varies, depending on the object of the study. In a cross-sectional study, relatively little 
time is required. In a longitudinal study, months or perhaps even years may be required. 

WHAT IS THE NATURE OF THE REAL WORLD? 

Bruner^(6) discusses two different kinds of concepts. The first, conjunctive concepts, 
consists of those in which the logic is essentially transitive. That is, if in Middletown only 
residents can vote, it is possible to infer by knowing that an individual has voted that he 
is a resident. Man-machine systems basically are conjunctive systems. The logic of 
man-machine interaction is a transitive logic. 

The second type of concept is disjunctive, in which this kind of transitivity does not 
occur. For example, if in Middletown either residents, taxpayers, or persons who work 
within the city may vote, then it is not possible *o infer the individual's category from 
knowing that he votes. It is very likely that most man-ascendant systems are 
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characterized by disjunctive logic, where several different solutions may work in any 
given problem situation. (This is recognized in the idiom by the statement that any 
solution, well implemented, is better than no solution.) 

Of course, it is possible that organizational leadership is not actually disjunctive. It 
may only appear so because of the many variables mat may affect the impact of any 
given leadership behavior or approach. Further, there may be moderator variables that 
actually change the relationship between a given leadership behavior and the expected 
outcome. (Fiedler's contingency model illustrates the use of moderator variables in 
leadership theory building.) Several studies illustrate these variables. 

Organizational Climate . In a relatively well-known study, Fleishman (7) reported the 
effects of leadership training for foremen after the foremen had returned to their 
industrial situation. The leadership training itself consisted of a focus on initiating 
structure and showing consideration dimensions. Fleishman reported that the effects of 
the training appeared rather small when a later follow-up was made. The kind of 
supervisor (leadership climate) under which the foreman worked was more related to his 
own behavior than either training or lack of it. Foremen working under a considerate 
supervisor tended to express more considerate attitudes toward their workers, and 
received, in return, higher consideration scores from their workers. 

Organizational Structure . Forehand and Gilmer (8) further analyzed the effect of 
organizational climate on leader behavior. They noted that climate may be determined 
not only by the leadership pattern of ore's own supervisors, but also by size, organiza- 
tional structure, systems complexity, and goals. Presumably, any of these factors could 
negate or enhance the impact of leadership training, or modify the relationship between 
given leadership behavior and system outcomes. 

Nonlinear Relationships. A report of a seminar held by the Foundation for Research 
on Human Behavior (9) describes several of these non-linear relationships: 

Production Pressure . When production pressure was low, an increase in pressure 
increased both satisfaction with supervisor and production, to a point, beyond which 
satisfaction dropped sharply. 

Group Cohesiveness . Group cohesion is positively correlated mth productivity 
when the company is perceived as supportive, but negatively correlated with productivity 
when the company is perceived as threatening. (This illustrates a moderator variable.) 

Closeness of Supervision . Production can be high either way, but morale suffers 
with excessively close supervision, and a large investment of supervisor energy must be 
expended also. 

Subordinate Expectations. Jasinski (10) reported an interesting study from the 
automobile industry, in which workers and foremen alike had proper and idealized 
descriptions of what the supervisor's role ought to be. However, in actual practice the 
foremsin's behavior was very different from the verbal description of the ideal role. 
Further, Jasinski found that workers really expected a different type of behavior from 
the foreman than they said they wanted, and the work environment precluded the 
lengthy interaction between workers and foremen that would have been required by 
"desired" behavior. Finally, the supervisors of the foremen gave little weight to foreman 
relations with men in their evaluations of the foremen. 

When leadership is taught and evaluated in a service school, the service school 
essentially is serving a customer. The customer is the commander in the field who uses 
the product generated by the school. It clearly is important to develop and evaluate the 
right thing. One further example of the cataclysmic results that may occur when the 
wrong thing is developed is found in a report by Sikes (11) of a course on human 
relations. This course was given to all managers and supervisors in a medium-sized 
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contracting firm. The evaluation tool was a questionnaire to supervisors. One question 
was, "Was the course as a whole a success or a failure"? The summary of answers was: 

Success 0 

Failure 83 

No Opinion 14 

These startling findings occurred because participants felt the course had not made 
any lasting improvement in the attitudes of senior management. However, the cataclysmic 
further result was a crisis in the conflict betw» n expectations held by supervisors and by 
senior management concerning the role of management. This conflict led to resignations 
of 19 of the 97 supervisors who had attended the course. (In the two years preceding the 
course only two had left.) Further, 25 more were known to have applied for other jobs. 



HOW TO MEASURE THE UNMEASURABLE? 

At the outset, it was noted that ultimate criteria are rarely available for measuring 
the impact of leadership training and development. There are probably two reasons. The 
first is the difficulty of obtaining objective criterion measures in real organizations; 
realistically, this does present problems. The second difficulty is that it sometimes is 
embarrassing to the researcher to be able to obtain such criteria when they are available. 

Korman (12) distinguishes two kinds of validity, concuirent and predictive. Con- 
current valiiiity is derived from measurements taken ai the same time that measurements 
are made of the so-called leader manipulation. Predictive validity is derived from a 
longitudinal study, in which a leadership manipulation is made, and efforts are then made 
to determine whether individuals in the experimental group behave in different (hopefully 
better) ways on the job. 

The embarrassing aspect of such criteria is that they frequently lead to the con- 
clusion Lhat no change has occurred, especially where the criteria consist of objective 
measures of organizational performance. It will be recalled that Fleishman's finding of no 
impact came from a longitudinal study. In his own review of the literature, Korman 
found quite similar results. His focus was on Initiating Structure and Showing Consid- 
eration as leadership behavior variables. Most of the studies reported concurrent validity, 
which Korman notes does not give good evidence of causality. Further, the objective 
criteria, when they were used, yielded low correlations with both Consideration and 
initiating Structure. This is a finding that is, of course, repeated fairly frequently in 
other studies. 

Korman 's other results are interesting in theu jwn right. Table 1 summarizes his 
review. It shows correlations between Initiating Structure and Showing Consideration 
variables, and a variety of other measures. In this table concurrent and predictive validity 
data have been mixed. As can be seen, there is substantial variability in the findings, 
regardless of v/hat criterion is used. The only consistent findings appear to be correlations 
between subordinate ratings of leaders and Showing Consideration behavior. Korman 
concludes that his survey of the literature does not provide any reason for saying that 
these leader behavior variables can predict work group performance, or under what 
conditions they might affect work group performance, a somewhat dismal picture. 

These findings represent only a marginal sampling from \ the available literature. 
However, they do nonetheless illustrate the extreme difficulty of determining what should 
be taught as leadership, how it should be taught, and how the effectiveness of the 
resulting skill should be evaluated. 
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Table 1 



Correlations Between Various Criteria and 
Initiating Structure and Showing Consideration 





tniiiatipg Structure 


Showing Consideration 


With 


N 


Range 


N 


Range 


Objective Measures 


5 


-.05 to ,23 


5 


-.11 to .14 


Superior Ratings 


8 


-.39 to. 13 


8 


-.06 to .29 


Peer Ratings 


8 


-.11 to .14 


8 


-.17 to .03 


Self Ratings 


3 


-.41 to ,45 


3 


-.02 to -.46 


Subordinate Hatings 


3 


.05 to 22 


3 


.24 to .51 



IF NOT ULTIMATE CRITERIA, THEN WHAT? 

While so-called ultimate criteria may be available within organizational settings, they 
generally are not available to th^ trainer. This has led to a proliferation of intermediate 
criteria which have been used for assessment purposes. Some of the vehicles for 
leadership assessment have been: 

SituaJonal Performance Tests 

Situational Paper-and-PencU Tests 

Personality Tests 

Peer Ratings 

Ratings by Seniors 

Leadership Grades 

This list is by no means inclusive. It is intended primarily to illustrate the diversity of 
approaches in current use. 

Situational Tests . These tests generally depend for predictive validity on the 
similarity bet^reen the performance requirement (including group leadership and 
diiection) of, tne test situation and that of the ultimate performance situation. However, 
situational tests have been plagued by many common psychometric problems, such as: 

• The difficulty of establishing parallel situations, so that different leaders will 
be assessed under comparable conditions of performance. 

• The difficulty of establishing situations which in fact present real life per- 
formance requirements at other than a face valid level. 

• The cost in terms of time and administrative difficulty in running good 
situational tests. 

• The difficulty of training assessors so that they respond to the same leader 
behaviors in the same way, or even obser\^e the same leader 
behavior variables. 

• The difficulty of providing subordinates who respond as subordinates would 
in real life. {In the test situation they are not subject to uuite the same 
actual leader power as in a real unit.) 

Situational Paper-and-Pencil Tests . An example is thd In-Basket Test. A problem 
with this kind of test is that one of the most critical dimensions of leadership skill 
probably is interpersonal competence. The austere context of an In-Basket Test is hardly 
appropriate for assessing this kind of variable. Further, in the military setting, there may 
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be some degree of stress associated with the performance requirement, which is difficult 
to simulate in an in-basket or other similar test. Finally, it is impossible to assess group 
maintenance or individual support leader behavior skills in a **solo** performance. 

Personality Tests . Typical of this approach, which is abouc three decades old, is the 
self-assessment test, which commonly takes the form of measuring the individuaPs 
proclivities toward authoritarian, democratic, or laissez-faire leadership. The fallacies of 
this approach are obvious. 

Peer Ratings. While it is difficult to know exactly what peer ratings measure, they 
generally have had better predictive validity than almost any other single approach. This 
may be because they probably measure the dimension of interpersonal competence and 
the ability to work within a coordinated group better than do most other approaches. 
However, even so, peer ratings rarely account for more than 25% of the total variance in 
subsequent leadership performance, and are always subject to the vagaries of the peers 
who are completing them. If a group decides to do so, ''cooperate and graduate" can be 
made to work extremely well. Nonetheless, peer ratings are a **method of choice," 
especially when used in conjun^-tion with other measures. 

Ratings by Seniors . It is very difficult to assess the extent to which ratings by 
seniors are predictive in an independent sense. That is, in a formal school setting, peer 
ratings and tactical officer ratings are found to be fairly highly correlated. However, 
ratings by tactical officers are generally less reliable than ratings by peers. 

Leadership Grades. Finally, leadership grades have been developed from paper- 
and-pencil knowledge tests following leadership instruction. In unpublished research, 
HumRRO Division No. 4 has found them on occasion even to be negatively correlated 
with leadership grades obtained through a combination of peer ratings and tactical officer 
ratings. This suggests that they probably would not predict subsequent performance, in 
that peer ratings generally do. 

A DISJUNCTIVE APPROACH FOR A DISJUNCTIVE CONCEPT 

The picture to this point has been negative, and unfairly so. It was presented in this 
fashion to illustrate the difficulties in both soft skills systems engineering, and the 
evaluation of soft skills such as leadership. Soft skills systems engineering is extraor- 
dinarily difficult because of the absence of a "true expert," and because of the intangible 
nature of these skills. Because the evaluation process is necessarily dependent on identi- 
fication of what is to be evaluated, the development of soft skill evaluation procedures 
has necessarily been impeded by the same problems. 

However, an assertion may be made at this point. While it is extremely unlikely that 
any massive breakthrough is ever going to occur in soft skills systems engineering, 
approaches such as the behavioral model (Whitmore (2)) are conceptually sound, and will 
eventually produce a sound picture of what it is that needs to be trained. It will then 
become feasible to develop evaluation techniques. The requirement is for patience and 
continued effort, with the realization that this is simply a much more difficult problem 
than hard skills systems engineering. 

As an example of possible future evaluation approaches, it is appropriate to examine 
the "scientific assessment center." Management assessment centers are becoming more 
frequent in industry, probably because, as Byham (13) asserts, they have shown them- 
selves . . to be a .more reliable indicator of future success than any other tool 
yet devised." 

The assessment center works, probably, because it is so comprehensive in what it • 
measures. (Critics might argue that if enoagh of an individual's behavior is observed, 
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probably it would be possible to make predictions to any future activity.) Byham 

identifies 26 ''common dimensions** of managerial success: 

Impact Listening skill 

Energy Flexibility 
Oral communciation skill Tenacity 
Oral presentation skill Risk taking 

Written communication skill In'\iative 
Creativity Independence 
Range of interest Planning and organizing 

Stress tolerance Management control 

Motivation Use of delegation 

Work standards Problem analysis 

Leadership Judgment 
Salesmanship Decisiveness 
Sensitivity 

Byham also identifies eight techniques that may be used in an assessment center for 
evaluating individuals on these dimensions: 
Interview 

Management Games 

In-Basket and Interview 

Leaderless Group Discussion (Assigned) 

Leaderless Group Discussion (Non-Assigned) 

Fact Finding and Decision Making 

Analysis Presenvation 

Interview Simulation 

Some may be better than others for assessing one or more of the preceding dimensions. 
In the assessment center, the optimum measurement approach is selected for each 
measurement to be obtained. 

According to Byham, psychologists are considerably superior to untrained line 
assessors in the assessment center context. However, they are not always available, so 
companies tend to establish pools of trained assessors. With either, the predictive validity 
of the assessment center is quite good. Bray, Grant, and Campbell (14) make an 
essentially similar point. 

The difficulty with assessment centers is, of course, that they are expensive to 
operate, and have a low throughput of assessed potential leaders. Where management 
turnover is low, this is no problem. In the Army, with high input of junior leaders, it 
might create difficulties. 

However, in the context of the present paper, the assessment center concept is 
probably worth examining as the prototype for soft skills evaluation procedures develop- 
ment. The assessment center "covers the waterfront." Similarly, it is probable that the 
evaluation of any soft skill will necessarily cover a multitude of dimensions. Single 
unitary measures probably will not suffice. (This suggests that leadership training also will 
need to cover a variety of dimensions.) Finally, it is likely that leadership evaluation 
procedures will gradually improve with experience and research, just as leadership 
development and training procedures improve through the same means. In each case, 
continued effort should provide increasingly improved payoffs. 

As a final example of the inextricable intertwining of leadership research and 
leadership development, it is appropriate to examine a probable new direction in the 
assessment of organizational leadership. Students of small group theory will affirm that 
substantial progress in understanding of small group dynamics came only after researchers 
turned to the study of processes that occur in small groups. Examination of the 



8 



assessment dimensions just cited shows that they are essentially person-oriented. Thus, 
they are likely to be limited in their predictive power. 

However, a few researchers have been turning to a study of organizational processes 
in order to understand organizational leadership more fully. This generally requires some 
sort of simulation of an organization, because real organizations cannot be "replayed." 
The capacity to repeat a standardi?:ed experience is, of course, necessary in order to 
develop a basis for inferential testing. As an example of this work, Olmstead (15) has 
developed a simulation of a battalion engaged in a counterinsurgency operation. The 
simulation is designed to permit measurement of organizational process variables 
identified by Schein (16) as the Adaptive Coping Cycle. These variables and their 
correlations with independent measures of the adequacy of mission accomplishment 



(Olmstead 's simulation) are: 

Sensing a change in the internal or external environment .92 

Communicating the information to a decision maker .83 

Making a decision with the information .70 

Stabilizing internal changes .11 

Communicating change decisions .71 

Coping actions .72 

Obtaining feedback as to decision effectiveness .03 



It is beyond the scope of the present paper to present Olmstead findings in great^^r 
depth. The significant point is that organizational leadership possibly should be viewed as 
the capacity to cause certain crucially important functions to be performed w^ll. This 
would have equally significant implications for assessment of leadership. The question 
would be, ''Leadership for what?" And the a-'^^sSiiient technique would be examination 
of the leader's capacity for making processes worl well. 

As a contrast, Olmstead also obtained dat? using many of the conventional paper- 
and-pencil measures, with miniscule results. In this simulation, the so-called "personality" 
measures used did not tap the capacity to make the process go smoothly. Also, they did 
not tap organizational effectiveness. The implications of these findings for assessment of 
leadership skills are profound. 



SUMMARY 

In this paper, it has been asserted that the development of leadership evaluation 
procedures has been impeded by the same obstacles that have slowed the development of 
soft skills systems engineering techniques. It is difficult to find a "true expert,*' and the 
incumbent is frequently unable to verbalize what he actually does because what he does 
is intangible. This means that knowledge must be developed through the arduous method 
of experimental study of leadership processes by researchers and other observers. How- 
ever, the general unavailability of ultimate criteria of leadership effectiveness makes this a 
slower process than it otherwise might be. 

The dearth of ultimate criteria has led to a proliferation of intermediate criteria. The 
researcher's own bias has often influenced the intermediate criteria he has chosen to 
study, and the hypotheses he has chosen to test. Researcher bias has also influenced the 
types of leadership processes he has chosen to study. 

Despite the handicaps that shackle the researcher, among which is the fact that he 
rarely has been in the position of a line executive and thus does not know how it feels, 
the researcher probably is indispensable in the soft skills area, as Whitmore (2) has noted. 
This is not because of what he knows substantively. Indeed, he may be more dangerous 
because of his substantive knowledge. Rather, the researcher's value stems from his 
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training in scientific methodology r^nd from the fact that he is, by virtue of that training, 
a highly qualified observer— providing that his own theoretical biases do not blind him. 

It is probable that most soft skill areas are disjunctive in nature. This creates 
substantial difficulties for anyone attempting to do systems engmeering work. However, 
substantial progress has been made in recent years in the development of knowledge 
about leadership, as an example of one soft skilK The assessment center concept was 
discussed briefly as a possible prototype of future evaluation approaches. 

Although not identified specifically in the paper, there are needs that are 
worth citing: 

(1) Continued research to define the substantive nature of the leadership 
phenomenon. Such research will continue to produce progress in the development of 
leadership training, and will permit parallel progress in the development of evaluation 
techniques. 

(2) Continued search for intermediate criteria. This will require the availability of 
organizations for study, perhaps through simulation techniques such as those Olmstead (15) 
has used. Further, there is a need for erJ^htened decision makers who will tolerate other 
than face-valid measures for leadership assessment. 

(3) Continued work to le£um how to modify an individual to be more effective 
as a leader. 
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